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This  is  a  Special  Report  prepared  for  Headquarters,  Air  Force  Logistics  Command. 


EXECITIVE  SUMMARY 


Objective 

This  report  provides  a  utilization  assessment  of  the  Technical  Order  (TO)  Managers  Handbook  prepared  by  the  Air 
Force  Human  Resources  Laboratory  (AFHRL). 


Requirement 

Request  for  Personnel  Research  (RPR)  77-20,  Development  of  Guidelines  and  A  Handbook  for  Selection  of  Format 
Options  for  Procurement  of  Technical  Data,  established  the  requirement  for  development  of  the  TO  Managers  Handbook. 
The  requirement  was  established  by  Headquarters,  Air  Force  Logistics  Command.  The  objective  of  the  research  and 
development  (R&D)  to  support  RPR  77-20  was  to  develop  a  handbook  that  would  be  useful  to  TO  managers  and  other 
personnel  concerned  with  the  acquisition,  development,  and  management  of  technical  data  for  the  Air  Force. 


Products 

Three  AFHRL  technical  reports  (TRs)  resulted  from  this  R&D  effort,  of  which  a  TO  Managers  Handbook  is  the 
primary  product.  Documentation  of  the  R&D  is  contained  in  AFHRL-TR-80-49,  Format  Options  and  Procurement  of 
Technical  Orders.  The  handbook  is  organized  into  two  volumes:  AFHRL-TR-80-50,  Technical  Order  Management  and 
Acquisition,  contains  the  basic  text,  and  AFHRL-TR-80-51,  Technical  Order  Managers  Reference  Data,  includes  the 
reference  data  and  supplementary  material. 


Specifics 

AFHRL  received  RPR  77-20  as  a  validated  requirement  in  November  1977.  The  requirement  was  in  response  to 
a  need  for  TO  acquisition  and  management  guidelines.  Active  support  of  RPR  77-20  by  the  AFHRL  Logistics  and  Human 
Factors  Division  began  in  May  1978.  The  resulting  TO  Managers  Handbook  provides  guidelines  for  selection  of  format 
options  and  procurement  of  Air  Force  TOs,  with  emphasis  on  the  responsibilities  involved  in  their  acquisition.  RPR 
77-20  was  completed  on  27  August  1982. 


Assessment  Results 

Fight  hundred  copies  of  the  handbook  were  printed.  Since  its  publication  in  May  1981,  the  handbook  has  been 
distributed  to  Air  Force  organizations  having  responsibilities  in  TO  acquisition  and  management.  A  number  of  these 
organizations  were  contacted  either  by  personal  interviews,  telephone,  and/or  letter,  followed  by  an  assessment  survey 
concerning  utilization  of  the  handbook.  A  primary  use  of  the  handbook  has  been  in  the  training  environment.  The  Air 
Force  Institute  of  Technology  (AFIT)  of  Systems  and  l-ogistirs  used  the  handbook  as  a  major  source  of  information  for 
the  development  of  a  TO  acquisition  and  management  course.  The  handbook  has  been  used  as  a  training  aid  by 
organizations  that  provide  their  own  training  in  TO  acquisition  and  management.  The  handbook  is  also  a  good  source 
of  reference  material,  particularly  for  TO  managers  with  limited  experience.  Opinions  regarding  the  content  of  the 
handbook  are  varied.  Some  felt  that  the  handbook  was  well  written,  concise,  accurate,  and  complete;  others  fell  the  exact 
opposite.  Many  felt  the  handbook  should  be  revised  and  updated  on  a  regular  basis  because  of  frequent  changes  in  TO 
policy  and  guidelines. 


Conclusion*  and  Recommendation* 

The  extent  to  which  the  handbook  is  a  useful  document  to  a  TO  manager  depends  on  the  individual’s  past  experience, 
level  of  expertise,  and  program  needs.  Generally,  the  more  knowledgeable  and  experienced  individuals  considered  the 


handlmok  the  least  useful,  whereas  the  inexperienced  individuals  considered  the  handt>ook  very  useful.  The  handlmok 
therefore  has  liern  a  valuable  document  to  many  organizations  with  inexfierienced  TO  managers.  However,  the  extent 
of  the  handbook's  usefulness  is  limited  by  the  inaccuracies  and  deficienees  expressed  by  some  organizations.  These 
shortcommings  can  be  explained  in  part  by  frequent  changes  in  TO  requirements,  policies,  and  procedures.  Other 
contributing  factors  may  be  the  failure  to  have  knowledgeable  TO  managers  develop  the  handbook  and  insufficient 
attention  given  to  the  user's  needs.  In  order  that  the  handbook  be  relevant  and  have  merit  as  an  effective  guide,  it  was 
recommended  that  it  lie  reaccomplished  by  skilled  Air  Force  TO  managers.  To  ensure  timely  revisions,  it  was  suggested 
that  changes  to  the  handbook  be  synchronized  with  revisions  to  an  official  TO  publication  such  as  TO  00-5-1 ,  AFTechnical 
Order  System. 


.  * -  - 
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TECHNICAL  ORDER  MANAGERS  HANDBOOK.:  UTILIZATION  ASSESSMENT 


I.  INTRODUCTION 

The  Air  Force  Technical  Order  (TO)  system  is  the  official  medium  for  disseminating  technical  information  and 
procedures  for  the  operation  and  maintenance  of  Air  Force  equipment  and  systems.  A  Technical  Order  Management 
Agency  (TOMA),  headed  by  a  TO  manager,  is  responsible  for  the  acquisition  of  technical  publications  for  the  operation 
and  maintenance  of  a  specific  program  or  item  of  equipment. 

Since  the  mid-1950s,  the  Air  Force  and  other  Department  of  Defense  (DoD)  agencies  have  performed  research  and 
development  (R&D)  to  improve  the  content  and  format  of  technical  data.  Several  types  of  improved  TO  formats  have 
resulted,  each  type  being  more  appropriate  for  some  applications  than  for  others.  The  R&D  efforts  have  demonstrated 
that  utilization  of  new  and  improved  TO  formats  has  a  tremendous  potential  to  improve  aircraft  maintenance  performance 
and  reduce  the  overall  cost  of  maintenance.  However,  even  though  DoD  approved  new  formats  for  use,  few  were 
implemented.  This  was  due,  in  part,  to  the  fact  that  many  TO  managers  were  not  sufficiently  familiar  with  the  new  formats 
to  select  and  apply  them  effectively.  A  readily  available  source  of  information  was  needed  to  assist  these  managers  in 
the  selection,  procurement,  and  development  of  TOs. 

The  need  for  appropriate  TO  guidelines  was  further  exemplified  by  the  fact  that  as  weapon  system  complexity  was 
increasing,  so  were  the  magnitude  and  variety  of  publications  required  to  support  a  weapon  system.  As  an  example, 
over  4,000  individual  TOs,  consisting  of  several  different  types,  are  required  to  support  the  F-15.  This  increase  in  TO 
requirements  has  been  very  costly.  In  May  1977,  HQ  USAF/LE  (Headquarters,  United  States  Air  Force,  Deputy  Chief 
of  Staff  for  Logistics  and  Engineering)  expressed  concern  over  the  alarming  increase  in  the  costs  of  TOs  for  new  weapon 
systems  compared  to  that  of  past  acquisitions.  An  example  was  cited  where  the  TO  requirement  for  a  weapon  system 
was  in  excess  of  one  million  pages  at  approximately  $300  per  page.  While  it  was  recognized  that  inflation  was  probably 
a  contributing  factor,  ineffective  TO  acquisition  management  was  also  suggested  as  a  factor  adding  to  the  increasing 
costs.  The  current  cost  estimate  for  procuring  TOs  for  the  B-l  is  $500M  at  $500  per  page  which  further  illustrates  the 
continuing  problem  and  the  magnitude  of  TO  acquisition  costs. 


II.  RAD  HISTORY 

Headquarters,  Air  Force  Logistics  Command  (HQ  AFLC)  is  responsible  for  the  management  and  operation  of  the 
TO  system.  This  includes  establishing  TO  requirements,  determining  TO  distribution  requirements,  reviewing  material 
content,  and  evaluating  new  methods  of  technical  data  presentation.  In  response  to  the  apparent  need  for  TO  acquisition 
and  management  guidelines,  HQ  AFLC/LOLMP  (now  MMEMB,  Deputy  Chief  of  Staff  for  Materiel  Management, 
Directorate  of  Engineering  and  Maintenance  Policy),  initiated  Request  for  Personnel  Research  (RPR)  77-20, 
Development  of  Guidelines  and  A  Handbook  for  Selection  of  Format  Options  for  Procurement  of  Technical  Data.  RPR 
77-20  was  submitted  to  the  Air  Force  Human  Resources  Laboratory  (AFHRL)  as  a  validated  requirement  in  November 
1977. 

The  objective  of  the  R&D  to  support  RPR  77-20  was  to  develop  a  handbook  that  would  be  useful  to  TO  managers 
and  other  personnel  concerned  with  the  acquisition,  development,  and  management  of  technical  data  for  the  Air  Force. 
The  guidelines  developed  by  this  effort  were  to  provide  data  and  procedures  for  requirements  determination;  format 
selection;  and  acquisition,  development,  and  implementation  of  Air  Force  TOs. 

Active  contractor  R&D  support  of  RPR  77-20  was  begun  in  May  1978  under  Contract  F33615-78-C-0016,  with 
work  unit  1710-04-31.  The  products  that  resulted  from  the  R&D  effort  consisted  of  the  following  AFHRL  Technical 
Reports  (TRs): 

1.  Hatterick,  G.R.,  &  Price,  H.E.  (May  1981a).  Formal  Options  and  Procurement  of  Technical  Orders.  AFHRL- 
TR-80-49,  AD-A099  448.  Wright-Patterson  AFB,  OH:  Logistics  and  Technical  Training  Division,  Air  Force  Human 
Resources  laboratory. 
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2.  Hatterick,  G.R.,  &  Price,  H.E.  (May  1981b).  Technical  Order  Management  and  Acquisition.  AFHRL-TR-80- 
50,  AD-A099  705.  Wright-Patterson  AFB,  OH:  Logistics  and  Technical  Training  Division,  Air  Force  Human  Resources 
Laborat  ry. 

3.  Hatterick,  G.R.,  &  Price,  H.E.  (May  1981c).  Technical  Order  Managers  Reference  Data.  AFHRL-TR-80-51, 
AD-A099  779.  Wright-Patterson  AFB,  OH:  Logistics  and  Technical  Training  Division,  Air  Force  Human  Resources 
Laboratory. 

Documentation  of  the  R&D  that  led  to  the  development  of  the  guidelines  for  selection  of  format  options  and 
procurement  of  TOs  is  contained  in  AFHRL-TR-80-49.  The  handbook  for  TO  managers  is  organized  into  two  volumes: 
AFHRL-TR-80-50  (basic  text)  and  AFHRL-TR-80-51  (reference  data  and  supplementary  material).  The  two  volumes 
may  be  used  together  as  a  basic  reference  publication  or  may  be  used  independently.  Some  data  are  duplicated  in  the 
two  volumes  to  support  their  independent  use. 

The  TO  Managers  Handbook  discusses  two  principal  topics  of  importance: 

1 .  The  Air  Force  TO  acquisition  cycle,  including  responsibilities,  methods,  and  data. 

2.  The  types  and  characteristics  of  procedural  data  formats  to  be  considered  for  use  in  Air  Force  maintenance 
manuals,  and  guidance  in  selection  of  the  most  appropriate  techniques  (Hatterick  &  Price,  1981a). 

The  organization  and  contents  of  the  TO  Managers  Handbook  is  presented  in  Table  1 . 

Of.  DEVELOPMENT  COSTS 

The  requirement  for  development  of  the  TO  Managers  Handbook,  RPR  77-20,  was  closed  out  on  27  August  1982. 
Job  Order  Cost  Accounting  System  (JOCAS)  data  indicated  that  the  total  cost  of  the  effort  was  $210,914.  The  R&D  was 
conducted  by  the  AFHRL  Logistics  and  Technical  Training  Division  (now  the  Logistics  and  Human  Factors  Division) 
forHQAFLC. 

IV.  ITIUZATION 

Methodology 

A  number  of  organizations  having  responsibilities  concerning  the  acquisition,  development,  or  management  of  TOs 
was  contacted  from  March  to  August  1983  to  determine  utilization  of  the  TO  Managers  Handbook.  A  comprehensive 
listing  of  the  organizations  providing  utilization  information  is  shown  in  Appendix  A.  The  organizations  were  contacted 
either  by  personal  interviews,  telephone,  and/or  letter,  followed  by  the  R&D  Product  Utilization  Assessment  survey 
developed  by  AFHRL/AZ,  Applications  and  Liaison  Office  (now  part  of  AFHRL/XR,  Plans  and  Programs  Office).  A 
copy  of  the  survey  is  shown  in  Appendix  B.  A  consolidation  of  the  views  on  the  handbook  is  included.  Footnotes  are 
used  to  attribute  utilization  statements  to  specific  user  organizations.  The  footnote  key  is  shown  in  Appendix  C. 

Product  Implementation 

The  handbook  is  intended  for  use  by  all  individuals  involved  in  the  acquisition,  development,  and  management 
of  Air  Force  TOs,  with  emphasis  on  the  responsibilities  and  concerns  of  the  TOMAs  in  those  commands  acquiring  TOs. 
Since  its  publication  in  May  1981,  800  copies  of  the  handbook  have  been  distributed  to  Air  Force  organizations  with 
responsibilities  for  TO  acquisition  and  management.  Although  the  list  is  not  exhaustive,  organizations  receiving  the 
handbook  include  HQ  USAF,  Air  Force  Institute  of  Technology  ( AFIT),  AFLC  locations  such  as  Air  Logistics  Centers 
(ALCs)  and  the  Air  Force  Acquisition  Logistics  Division  (AFALD),  Air  Force  Systems  Command  (AFSC)  locations  such 
as  the  Electronics  Systems  Division  (ESD)  and  Systems  Program  Offices  (SPOs)  within  the  Aeronautical  Systems  Division 
(ASD),  the  Tri-Service  Working  Committee  on  Technical  Data,  and  the  Air  Force  TO  Council  (now  the  USAF  Centralized 
Technical  Order  Management  (CTOM)  Group),  which  included  representatives  from  Major  Commands  (MAJCOMs). 


Table  1 .  Organisation  and  Contents  of  TO  Managers  Handbook 


AFHRL-TR-80-50:  Technical  Order  Management  and  Acquisition 

Section 

General 

□ 

1.  Overview 

Information 

2.  The  Air  Force  Technical  Order  System 

3.  Maintenance  Procedure  Format  Options-General 

4.  Format  Options:  Job  Performance  Aid  Systems 

Format 

5.  Format  Options:  Primary  Formats  and  Formats 

6.  Format  Option  Selection  Guidelines 

7.  Familiarization  of  Users  With  New  Formats 

8.  Technical  Order  Requirements  Determination 

TO 

9.  Technical  Order  Acquisition  Management 

Acquisition 

1 0.  Quality  Assurance  of  Technical  Orders 

1 1 .  Technical  Order  Publication,  Distribution,  and 

— 

Revision 

Bibliography, 

12.  Bibliography  of  Reference  Publications 

Glossary, 

13.  Acronyms  and  Abbreviations 

and  Index 

14.  Topical  Index 

AFHRL-TR-80-51 :  Technical  Order  Managers  Reference  Data 

Section 

Glossaries 

□ 

1 .  Glossary  of  Terms 

2.  Abbreviations  and  Acronyms 

JPA 

3.  Job  Performance  Aid  Research  and  Development 

and  Format 

4.  Job  Performance  Aid  Concepts  and  Techniques 

Data 

— 

5.  Format  Test  and  Evaluation  Data 

6.  Format  Option  Selection  Data 

Specifications 

7.  Technical  Manual  Requirements  and  Guidelines 

and 

Documents 

Standards 

8.  Requirements  Documents  Cross-Reference  Tables 

9.  Evaluation  of  Military  Technical  Manual 

— 

Specifications 

Bibliography 

c 

10.  Bibliography  of  Reference  Publications 

and  Index 

11.  Topical  Index 

Note.  Extracted  from  Hatterick  A  Price.  1961a. 
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The  handbook  appears  to  be  used  to  the  greatest  extent  by  TO  managers  with  limited  experience. 1  There  are, 
however,  experienced  managers  who  also  use  the  handbook.2  A  primary  benefit  provided  by  the  handbook  is  that  it  is 
a  good  source  of  reference  material.3  Specifically,  the  handbook  is  a  quick  reference  for  terminology,  acronyms,  cross- 
references  to  other  documents,4  and  options  a  TO  manager  has  available  in  the  acquisition  of  TOs.5  One  organization 
stated  that  by  having  the  handbook  as  a  ready  source  of  reference,  their  operational  effectiveness  was  increased.6 

The  Air  Force  Institute  of  Technology  (AF1T)  of  Systems  and  Logistics  located  at  Wright-Patterson  AFB  initiated 
a  2-week  course  of  instruction  in  TO  acquisition  and  management  in  May  1982.  The  course  (SYS  010,  Introduction  to 
Acquisition  Management)  was  designed  to  provide  continued  professional  education  to  Air  Force  personnel  with  duties 
in  the  acquisition  and  management  of  TOs.  The  handbook  was  a  major  source  of  information  for  the  development  of  the 
course.7  The  information  presented  in  the  course  is  directly  referenced  to  official  Air  Force  directives  concerning  TO 
acquisition  and  management;  however,  approximately  85%  of  the  information  presented  in  the  course  is  contained  in 
the  handbook.8  A  copy  of  the  handbook  is  provided  to  each  student  attending  the  course.  The  students  have  commented 
favorably  concerning  the  content  of  the  handbook,  especially  in  regard  to  the  sections  listing  reference  documents  and 
definitions  of  acronyms.9 

A  joint  AFLC/AFSC  survey  discovered  that  it  took  a  minimum  of  6  months  to  1  year  to  become  knowledgeable  in 
TO  acquisition.  Many  organizations  therefore  provide  their  own  training  in  TO  acquisition  and  management  and/or 
supplement  the  training  by  A  FIT,  which  is  not  a  mandatory  course.  The  handbook  is  very  beneficial  as  a  training  aid 
to  these  organizations. 10  One  organization  stated  that  use  of  the  handbook  probably  reduced  their  total  amount  of  training 
time  by  at  least  1  month. 1 1 

There  are  other  organizations  that  do  not  provide  TO  managers  with  any  specific  type  of  formal  training.  These  individuals 
are  required  to  learn  their  duties  as  a  TO  manager  through  on-the-job  training.  For  many  of  these  individuals,  the 
handbook  has  been  beneficial  as  a  source  reference  document. 12 

There  is  some  disagreement  among  users  regarding  the  content  of  the  handbook.  It  is  considered  by  some 
organizations  to  be  very  well  written  and  easily  understood.13  It  has  been  said  to  be  accurate,  complete,  and  concise 
and  addresses  most,  if  not  all,  areas  of  TO  acquisition  management. 14  However,  in  contrast,  other  organizations 
considered  the  handbook  to  be  poorly  written  and  not  easily  understood. 15  It  was  said  to  be  too  voluminous  to  be  an 
effective  guide. 16  Criticisms  were  that  there  is  too  much  history  and  background  information,17  in  addition  to  a  lot  of 
unnecessary  duplication  of  material  contained  in  other  sources. 18  Also,  some  organizations  felt  that  the  basic  needs  of 
a  TO  manager  were  inadequately  addressed. 19  Specific  examples  of  inadequacies  mentioned  include  TO  management 
concepts,  TO  management  structure  and  where/who  to  go  for  assistance,  and  a  listing  of  governing  documents  and  their 
purpose.20  Many  organizations  considered  the  information  contained  in  the  handbook  to  be  inaccurate  and  in  need  of 
revision  and  update.21  Some  of  the  organizations  that  were  critical  of  the  handbook  did,  however,  consider  Section  9, 
Technical  Order  Acquisition  Management,  of  AFHRL-TR-80-50  useful. 22 

V.  PAYOFF 

The  handbook  is  considered  by  many  organizations  to  be  a  good  source  of  reference  material.  It  is  useful  as  a  desk 
reference  and  as  a  training  aid,  particularly  for  inexperienced  TO  managers.  This  lack  of  knowledge  by  TO  managers 
is  a  common  problem  experienced  by  many  organizations  involved  in  the  acquisition,  development,  or  management  of 
TOs.  Their  prior  TO  experience  is  often  no  more  than  that  of  a  user  in  the  field  or,  at  best,  as  a  MAJCOM  participant 
in  the  TO  verification  process.  Some  of  the  factors  that  contribute  to  this  problem  are: 

1 .  No  mandatory  training  in  TO  acquisition  management. 

2.  No  specific  Air  Force  Specialty  Code  ( AFSC)  for  TO  managers. 

3.  Relatively  rapid  turnover  of  personnel,  particularly  military  personnel. 

4.  No  specific  qualifications  involved  in  the  assignment  of  TO  managers. 


-  '  • ' 
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As  a  result,  many  TO  managers  are  untrained  and  inexperienced.  Accordingto  one  organization,  approximately  50% 
of  their  present  workforce  is  staffed  with  •‘trainees.”2'*  Use  of  the  handbook  has  widespread  implications,  considering 
the  number  of  TO  managers  that  lack  sufficient  training  and  experience. 

The  handbook  was  also  a  very  useful  source  of  information  for  structuring  the  course  offered  by  AF1T  in  TO 
acquisition  and  management.  During  the  past  16  months,  approximately  250  students  have  attended  and  been  provided 
copies  of  the  handbook. 

A  lienefit  that  is  difficult  to  measure  is  the  impact  the  handbook  has  on  the  maintenance  environment.  The  quality 
of  technical  data  procured  should  be  enhanced  by  using  acquisition  guidelines  that  assist  a  TO  manager  in  selecting 
the  best  type  of  technical  data  available  for  a  given  application.  When  the  type  selected  meets  the  needs  of  a  specific 
weapon  system,  as  well  as  the  needs  of  the  user  in  the  field,  improved  maintenance  performance  and  lower  maintenance 
costs  should  result. 


VI.  CONCLUSIONS 

The  TO  Managers  Handbook  was  prepared  with  the  goal  of  serving  as  a  valuable  aid  in  all  aspects  of  TO  acquisition, 
development,  and  management.  The  broad  target  population  for  which  the  handbook  was  developed  explains  in  part  the 
different  opinions  expressed  regarding  its  value.  The  past  experience,  levels  of  expertise,  and  program  needs  of  each 
individual  have  a  direct  bearing  on  the  handbook's  usefulness.  Generally,  the  more  knowledgeable  and  experienced 
individuals  considered  the  handbook  the  least  useful,  whereas  the  inexperienced  individuals  considered  the  handbook 
very  useful,  particularly  as  a  quick  reference  document.  Besides  being  a  good  training  aid  for  organizations  with 
inexperienced  TO  managers,  the  handliook  was  very  useful  in  development  of  the  AFIT  course  in  acquisition 
management.  Therefore,  by  measuring  the  value  of  the  handbook  by  its  usefulness  to  the  TO  professional,  the  handbook 
can  be  considered  a  success.  However,  the  extent  of  the  handbook's  usefulness  is  limited  because  of  the  inaccuracies 
and  deficiencies  expressed  by  some  organizations.  These  shortcomings  can  be  explained  in  part  by  the  frequent  changes 
in  TO  requirements,  format  technology,  specifications.  Air  Force  policy,  regulations,  and  procedures.  For  the  handbook 
to  have  merit,  these  changes  need  to  be  incorportated  periodically.  It  has  been  suggested  that  the  handbook  be  published 
as  a  looseleaf  document  so  that  changes  can  easily  be  inserted.2*  To  ensure  timeliness,  these  changes  or  revisions  should 
he  synchronized  with  an  official  publication  that  deals  with,  or  provides  requirements  for  use  of,  the  TO  system,  such 
as  AFR  8-2.  TO  00-5-1,  TO  00-5-2.  or  TO  00-5-15.  Other  contributing  factors  to  the  handbook’s  shortcomings  may 
be  the  failure  to  have  knowledgeable  TO  managers  actively  participate  in  developing  the  handbook  and  insufficient 
attention  given  to  the  user's  needs.  Reaccomplishment  of  the  handbook  by  skilled  Air  Force  TO  managers  instead  of 
a  civilian  contractor  should  1  letter  ensure  that  the  handbook  addresses  the  needs  of  the  users.  This  should  also  satisfy 
most  of  the  specific  recommendations  obtained  during  this  assessment  as  to  what  topicsshould  be  included. 

F.ight  hundred  copies  of  the  handbook  were  originally  printed.  In  additionto  the  original  distribution,  AFHRL  has 
lieen  supplying  copies  to  organizations  on  an  as-requested  basis  and  to  AFIT  on  an  as-needed  basis  for  the  students 
attending  the  acquisition  management  course.  This  original  stock  is  practically  depleted,  and  there  is  no  provision  or 
plan  to  provide  additional  copies  or  a  revised  form  of  the  handbook.  In  addition  to  AFIT,  it  is  anticipated  that  other 
organizations  will  continue  to  request  copies  periodically.  As  an  example,  35  additional  copies  of  the  handbook  were 
provided  in  response  to  requests  from  various  organizations  that  were  contacted  for  this  utilization  assessment.  Unless 
AFl.C,  AFSC,  or  another  interested  agency  initiates  an  effort  to  reaccomplish  the  handbook,  its  use  and  value  as  an 
effective  guide  to  TO  managers  will  be  minimal.  If  AFl.C  feels  that  a  handbook  of  this  type  is  no  longer  warranted,  the 
primary  user,  AFSC,  should  consider  pursuing  its  reaccomplishment.  Since  the  majority  of  the  acquisition  funds  for  TOs 
are  provided  by  AFSC,  an  updated  handbook  that  meets  the  needs  of  TO  managers  could  have  a  substantial  positive 
impact  on  the  procurement  ofTOs  in  terms  of  quality  and  cost  savings. 


VII.  RECOMMENDATIONS 


The  organizations  contacted  recommended  the  following  topics  be  included  to  better  ensure  that  the  TO  Managers 
Handbook  provides  adequate  guidelines  in  the  acquisition,  development,  and  management  of  TOs.  (These  topics  arc 
illustrative  rather  than  exhaustive,  and  the  order  of  presentation  does  not  signify  their  degree  of  importance. ) 

1 .  What  to  procure  in  preproduction  equipment  that  must  be  supported  by  Air  Force  personnel. 2S 

2.  What  is  required  for  equipment  to  be  supported  by  contract  under  competitive  bid. 26 

3.  The  application  of  commercial  manuals  and  their  common  shortfalls.27 

4.  TO  verification  options  and  procedures  for  planning  a  TO  verification. 28 

5.  Interface  between  the  other  services.20 

6.  Types  of  multi-service  manuals. 1,0 

7.  Current  contractor  methods  regarding  TO  development  and  verification  processes.  '*1 

8.  Types  of  specifications  and  their  effectiveness.'12 

9.  The  role  of  AFLC  and  AF  ALD  in  acquisition  and  management  of  TOs.1'1 

1 0.  Adverse  effects  of  ineffective  TO  manning. 14 

1 1.  Results  and/or  cost  savings  of  applying  Military  Specification  MIL-M-82495,  Manuals.  Technical,  Organiza¬ 
tional  Maintenance  Manual  Set:  General  Requirements for  Preparation  of  Aircraft,  Missiles ,  and  Space  Vehicles. 1,7 

12.  Effectiveness  of  past  programs.'16 

The  following  general  recommendations  were  also  expressed: 

1 .  The  handbook  should  be  reorganized  to  better  assist  the  user  in  locating  specific  items  of  information.  *' 

2.  The  handbook  should  lie  reaccomplished  by  skilled  Air  Force  TO  managers  to  better  ensure  that  basic 
requirements  and  problems  encountered  by  a  TO  manager  are  adequately  addressed.18 

This  apparent  need  to  reaccomplish  the  handbook  is  further  accentuated  by  the  fact  that  as  TO  requirements, 
policies,  and  procedures  change,  the  handbook's  relevancy  diminishes.  An  approach  that  was  suggested  is  for  AFI.C 
(the  requirements  organization)  and  AFSC  (the  primary  user)  to  jointly  reaccomplish  the  handbook.10  This  joint  effort 
could  possibly  be  performed  by  the  Air  Force  Acquisition  Logistics  Center  (AFALC)  since  AFALC  is  a  joint  AFLC  and 
AFSC  enterprise  which  has  acquisition  logistics  responsibilities  for  the  Air  Force. 40  This  responsibility  includes  ensuring 
that  the  logistics  disciplines  are  applied  early  in  the  procurement  of  equipment  and  TOs.  To  further  expound  on  this 
approach  to  reaccomplish  the  handbook,  it  was  recommended  that  a  working  group  be  formed  consisting  of  representatives 
from  TOMAs  (AFSC)  and  representatives  from  the  technical  publication  offices  of  ALCs  (AFLC).  Once  reaccomplished, 
the  handbook  could  be  made  an  official  Air  Force  publication,  with  AFALC  as  the  Office  of  Primary  Responsibility 
(OPR).41  Future  revisions  of  the  handbook  could  be  synchronized  with  changes  to  an  official  TO  publication  that  assigns 
responsibilities  or  prescribes  specific  requirements,  such  as  AFR  8-2,  Air  Force  Technical  Order  System;  TO  00-5-1, 
AF  Technical  Order  System;  TO  00-5-2,  Technical  Order  Distribution  System;  or  TO  00-5-15,  Air  Force  Time 
Compliance  Technical  Order  System.  Foremost  among  these  publications  is  TO  00-5-1.  In  addition  to  identifying  and 
describing  the  various  types  of  TOs,  this  publication  describes  the  concept  and  management  of  the  TO  system  and  the 
specific  responsibilities  relating  to  the  acquisition  and  use  of  TOs.  To  ensure  success  of  the  handbook’s  reaccomplishment 
and  timely  revisions,  the  IJSAF  CTOM  Group  might  have  a  supervisory  role.  The  CTOM  Group,  which  consists  of 
representatives  frm  all  MAJCOMs,  provides  the  mechanism  for  management  and  improvement  of  all  facets  of  the  TO 
system.  In  this  capacity,  the  CTOM  Group  could  assist  in  selecting  well  qualified  TO  managers  for  the  working  group 
and  provide  advice  when  needed. 


APPENDIX  A:  ORGANIZATIONS  PROVIDING  UTILIZATION  INFORMATION 


Requirements  Organization 

HQ  AFLC/MMEMB  (formerly  HQ  AFLC/LOLMP):  Headquarters  Air  Force  Logistics  Command,  Deputy  Chief  of 
Staff  Materiel  Management,  Directorate  of  Engineering  and  Maintenance  Policy,  Maintenance  Engineering  Branch 

Additional  Organizations  Responding  to  Utilization  Assessment  Survey 

HQ  USAF/LEYE:  Headquarters  United  States  Air  Force,  Deputy  Chief  of  Staff  Logistics  and  Engineering, 
Directorate  of  Maintenance  and  Supply,  Acquisition  Logistics  Communications  Group 

HQ  AFSC/ALPG:  Headquarters  Air  Force  Systems  Command,  Deputy  Chief  of  Staff  Acquisition  Logistics, 
Directorate  of  Program  Readiness  and  Evaluation,  Program  Readiness 

AFALD/PTJ:  Air  Force  Acquisition  Logistics  Division,  Deputy  for  Engineering  and  Evaluation,  Logistics  Projects 
Office 

AFALD/PTJ:  Air  Force  Acquisition  Logistics  Division,  Deputy  for  Engineering  and  Evaluation,  Directorate  of 
Lessons  Learned,  Lessons  Learned  Application  Division 

AFIT/LSY:  Air  Force  Institute  of  Technology,  School  of  Systems  and  Logistics,  Department  of  System  Acquisition 
Management 

ASD/AEGL:  Aeronautical  Systems  Division,  Deputy  for  Aeronautical  Equipment,  Subsystems/Support  Equipment 
System  Program  Office,  Integrated  logistics  Support 

ASD/AESL:  Aeronautical  Systems  Division,  Deputy  for  Aeronautical  Equipment,  Life  Support  System  Program 
Office,  Deputy  Program  Manager  for  Logistics 

ASD/AFC:  Aeronautical  Systems  Division,  Deputy  for  Airlift  and  Trainer  Systems,  Directorate  of  Configuration  Data 
Management 

ASD/AFMCH:  Aeronautical  Systems  Division,  Deputy  for  Airlift  and  Trainer  Systems,  Director  of  Consolidated 
Data  Management  Information  Systems,  Directorate  of  Configuration  Data  Management  Division,  Airlift  Systems 
Branch  Configuration  Data  Management  Division 

ASD/AFYC:  Aeronautical  Systems  Division,  Deputy  for  Airlift  and  Trainer  Systems,  Directorate  of  Consolidated 
Management  Information  Systems,  KC-135/KC- 10  Systems  Division 

ASD/AWL:  Aeronautical  Systems  Division,  Deputy  for  Acquisition  Support,  Directorate  of  Logistics 

ASD/RWNL:  Aeronautical  Systems  Division,  Deputy  for  Reconnaissance/Strike  and  Electronic  Warfare  Systems, 
Directorate  of  Strike  Systems  Program  Office,  Deputy  Program  Manager  for  Logistics 

ASD/TAAXL:  Aeronautical  Systems  Division,  Deputy  for  Tactical  Systems,  Directorate  of  Fighter/ Attack  System 
Program  Office,  A*  10  Program  Division,  Deputy  Program  Manager  for  Logistics 

ASDH'AFC:  Aeronautical  Systems  Division,  Deputy  for  Tactical  Systems,  F-15  Systems  Program  Director, 
Configuration  and  Data  Management  Division 

AS  DAT  AMC:  Aeronautical  Systems  Division,  Deputy  for  Tactical  Systems,  AGM-65  (Maverick)  System  Program 
Director,  Configuration  and  Data  Management  Division 


ASD/YWLL:  Aeronautical  Systems  Division,  Deputy  for  Simulators,  Directorate  of  Integrated  Logistics  Support, 
Logistic  Support  Division 

ASD/Y  YL:  Aeronautical  Systems  Division,  Deputy  for  Strategic  Systems,  Directorate  of  Integrated  Logistics  Support 

ASD/YZYL:  Aeronautical  Systems  Division,  Deputy  for  Propulsion,  Strategic  Engines  Program  Office,  Strategic 
Engine  Program  Office 

ESDATCRL:  Electronics  Systems  Division,  Deputy  for  Tactical  Systems,  Directorate  of  Tactical  Air  Battle 
Management  Systems,  Logistics  Division 

AFLMC/LGM:  Air  Force  Logistics  Management  Center,  Directorate  of  Maintenance 

SA-ALC/MMEDTA:  San  Antonio-Air  Logistics  Center,  Directorate  of  Materiel  Management,  Deputy  for 
Engineering  Division,  Operations  and  Support  Branch,  Technical  Order  Data  Section,  AGE,  ATE,  and 
Miscellaneous  Commodities  Requirements  Center 

SM-ALC/MMAA:  Sacramento- Air  Logistics  Center,  Directorate  of  Materiel  Management,  Deputy  for  Acquisition 
Division,  Administrative  Support  Branch 

SM-ALC/MMEDT:  Sacramento-Air  Logistics  Center,  Directorate  of  Materiel  Management,  Deputy  for  Engineering 
Division,  Operations  and  Support  Branch,  Technical  Order  Data  Section 

ATC  3760TTG/TTGRB:  Air  Training  Command,  3760th  Technical  Training  Group,  Aircraft  Principles  Branch 

ATC  3785  FTG/3751  FTS/405  FTD:  Air  Training  Command,  3785th  Field  Training  Group,  3751st  Field  Training 
Squadron,  405  Field  Training  Detachment 


APPENDIX  B:  R&D  PRODUCT  UTILIZATION  ASSESSMENT  SURVEY 


AIR  FORCE  HUMAN  RESOURCES  LABORATORY 
Air  Force  Systems  Command 
Brooks  Air  Force  Base,  Texas  78235 
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plications  and  Liaison  Office  of  the  Air  Force  Human  Resources  Labo 
JAZ )  is  responsible  for  tracking  the  utilization  of  AFHRL’s  researcl 
ment  efforts.  Your  responses  are  needed  to  assess  the  utilization  of: 

Technical  Order  Managers  Handbook 

NT 
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AFHRL-TR-80-49 ,  AFHRL-TR-80-50,  AFHRL-TR-80-51 

Noie:  For  each  scaled  quest  ion,  please  circle  the  appropriate  number  and  comment  further  as  you 
desire.  If  necessary,  attach  additional  pages  for  your  comments. 


1.  To  what  extent  were  the  needs  of  your  organization  addressed  during  this 
R&D  effort? 


1 

i 

2 

I 

3 

i 

4 

1 

5 

1 

1 

Very 

Inadequately 

Comments: 

1 

Inadequately 

(Please  be  as  specific 

1 

Neutral 

as  you  can.) 

1 

Adequately 

1 

Very 

Adequately 

2.  How  useful  is  this  product  to  your  organization? 


1 

t 

2 

1 

3 

i 

4 

1 

5 

1 

1 

Not  at 

1 

Not  Very 

1 

Sufficiently 

1 

Very 

1 

Extremely 

All 

Useful 

Useful 

Useful 

Useful 

Comments:  (Please  be  as  specific  as  you  can.) 


All 


Little 

Extent 


Moderate 

Extent 


Great 

Extent 


Very  Great 
Extent 


Comments:  (If  the  product  is  not  being  used,  please  explain  why  this  is  the  case.  If 
it  is  used,  please  specify  any  concrete  benefits,  e.g.,  cost  savings,  increased 
operational  effectiveness,  improved  force  structure.  Also,  if  possible,  please 
include  the  time  required  for  implementation,  the  cost  of  implementation  and 
any  barriers  to  implementation/utilization.) 


4. 


Please  identify  (name/oflice  symbol)  other  organizations  that  currently  use  this 
product. 

_ Unknown 


5.  Please  specify  any  additional  possible  uses  for  this  product. 


Unknown 


6.  Please  identify  (name/office  symbol)  other  organizations  that  could  possibly  use 
this  produet. 

Unknown 


7.  What  follow-on  R&D  would  your  organization  consider  potentially  beneficial? 

Unknown 


8.  Please  comment  additionally  on  this  product  (relevance,  timeliness,  clarity,  completeness, 

etc.). 


Gl>  or  Raw 


Stale 
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'HQ  USAF/LEYE,  ASD/AEGL,  ASD/RWNL,  ASD/TAMC,  ASD/YWLL,  ASD/YYL,  SA-ALC/MMEDTA 
2ASD/YYL 

3HQ  AFLC/MMEMB,  AFIT/LSY,  ASD/AESL,  ASD/AFC,  ASD/AFYC,  ASD/RWNL,  ASD/TAMC,  ASD/YYL,  SA-ALC/ 
MMEDTA 

*HQ  USAF/LEYE,  AFIT/LSY,  SA-ALC/MMEDTA 
5  ASD/AFC,  ASD/AFMCH,  ASD/RWNL 
6ASD/AFC 
7AFIT/LSY 

8afit/lsy 

9afit/lsy 

loHQ  AFLC/MMEMB,  HQ  USAF/LEYE,  ASD/AFC,  ASD/YYL,  SA-ALC/MMEDTA 
"ASD/YYL 

l2HQ  AFLC/MMEMB,  ASD/AESL,  ASD/AFC,  ASD/AFYC,  ASD/RWNL.  ASD/TAMC,  ASD/YYL,  SA-ALC/MMF.DTA 
"AFIT/LSY.  ASD/AESL,  ASD/RWNL,  SA-ALC/MMEDTA 
"AFIT/LSY,  ASD/RWNL,  ASD/TAMC,  AFLMC/LGM,  3760TTGTTGRB 
"HQ  AFSC/ALPG,  ASD/TAFC 

"HQ  USAF/LEYE,  HQ  AFSC/ALPG,  ASD/AWL,  SM-ALC/MMEDT 

l7HQ  USAF/LEYE 

l8ASD/AWL,  SM-ALC/MMEDT 
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